Universidad Pontificia Comillas. Madrid (España)
12 de enero de 2026
Resumen:
Microgrids play a key role in the energy transition by improving grid resilience, enabling electrification in remote areas, and increasing renewable energy integration. By clustering distributed energy resources near loads, microgrids can operate independently or cooperatively, enhancing reliability during grid disturbances. However, their heavy reliance on intermittent renewables introduces operational challenges, making energy storage and effective coordination essential. Conventional Energy Management Systems (EMS) rely on complex optimization models that require accurate forecasts and frequent recalibration, which can be impractical for small-scale microgrids. As a result, simpler rule-based approaches are often used, despite their limited performance. Recent advances in Artificial Intelligence, particularly Deep Reinforcement Learning (DRL), offer a promising alternative. This thesis investigates DRL-based EMS design using two algorithms: Deep Q-Network (DQN) and Twin-Delayed Deep Deterministic Policy Gradient (TD3). By modeling EMS operation as a sequential decision-making problem, DRL agents learn control policies directly from interaction with the microgrid, reducing dependence on forecasting and manual tuning. Three studies are presented. Chapter 3 demonstrates that a DQN-based EMS can effectively manage an isolated microgrid using historical data, achieving near-optimal performance compared to an idealized optimization benchmark. Chapter 4 extends the approach to continuous control using TD3, yielding improved performance and precision in both small-scale and benchmark microgrids. Chapter 5 incorporates a nonlinear battery loss model, showing that TD3 can exploit more realistic dynamics to reduce battery losses and operational costs without significant computational burden. Overall, the dissertation shows that DRL-based EMS are adaptable, scalable, and well-suited for real-world microgrid operation. By avoiding heavy optimization and forecasting requirements, DRL enables more robust and autonomous energy management. Future work includes demand response, multi-agent coordination, and market participation for grid-connected microgrids.
Resumen divulgativo:
Esta tesis indaga en las ventajas que tienen las técnicas de aprendizaje por refuerzo profundo sobre sus alternativas clásicas de control óptimo, abordando la gestión de la energía en microrredes. Las contribuciones ahondan en resolver el control óptimo de horizonte infinito considerando dinámicas no lineales.
Descriptores: Matemáticas, Ciencia de Los Ordenadores, Inteligencia Artificial
Palabras clave: Deep reinforcement learning; Isolated microgrids; Energy management system; TD3; Nonlinear battery models
Cita:
C. Domínguez-Barbero, "Modeling and optimizing isolated microgrids using Reinforcement Learning techniques", Tesis Doctoral, Universidad Pontificia Comillas, Madrid, España, 2026.
